The approaches described in this section could be pursued, but are not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated herein, the approaches described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.
Video and audio calls and conferences are becoming both more common and larger in size every day. Originally, video and audio calls and conferences hosted using computers were limited to direct connections between users in which a first user called a second user, enabling the users to see and hear each other during the call. As methods improved, conferences of five, ten, or more users all with streaming video and audio became more common. Simply sending all video and audio data from each participant to every other participant in such a large conference often is not feasible due to limitations on network bandwidth in local area networks and internetworks and the limited processing power in some client devices.
In response, solutions to large video and audio conferences were implemented with server computers using two general approaches: multipoint control units (MCU) and selective forwarding units (SFU). MCUs receive and process audio and video data from clients and typically mix the audio and video to create a single composite video. The composite video is then streamed to the clients in a single mix. While this solution enables relatively simple client software, the processing power needed for the MCU is high, and the resulting presentation at the client is generally inflexible. SFUs, in contrast, are more similar to packet data routers, as SFUs forward packets without media processing.
Due to limitations on network bandwidth and processing power, SFUs may improve their efficiency by only sending a subset of the video streams to each computing device. In an extremely large video conference, sending all of the video streams to each of the computing devices in the conference would be egregious, as most of the video streams are likely unnecessary to the majority of the people in the video conference. Thus, the SFU may reduce the load on the SFU by limiting the number of video streams sent to each participant computing device in a conference.
While setting a hard limit on the number of video streams sent to each computing device in a conference effectively reduces the load on the SFU, a hard limit is not adaptable to different situations. For instance, there may be times when the load on the SFU is especially high, such as during peak conferencing hours or in the case of a particularly large conference. When the SFU begins overusing its available resources, the ability of the SFU to forward high quality videos begins to degrade. For example, when the SFU uses too much bandwidth, the SFU begins to lose packets over the network, resulting in bad video quality for all users of the SFU.
A solution to the overuse of network resources is to create stronger limits on network usage, such as by using a smaller limit on the number of video streams sent to each participant computing device in a conference. While this method would reduce the usage of network resources, it would also degrade the experience for members of the conference. In times of low system usage, each participant computing device would still be limited on the number of received video streams, even though the SFU has bandwidth to send a greater number of video streams.
Thus, there is a need for a method that allows an SFU to dynamically adapt to overuse and underuse of system resources while minimizing the negative effects on user experience.